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ABSIEACT 

The "Trouble Shooting" Checklist (TSC) for Higher 
Educational Settings has been developed to assist educational change 
agents, faculty, and administrators concerned nith change, ii^ their 
assessment of organizational variables predictive of a department's 
potential for successfully adopting .innovations. The higher , 
educational based TSC is both diagnostic and predictive. That, is, the 
TSC provides- a means of organizing information in a predictive way. 
In addition, the TSC contains distinct scales that identify ^ 
particular strengths and weaknesses within a department. This paper 
provides information on the developms-nt of the TSC, reliability, 
validity, and norms. (Author) 
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Preface 



Jhe "Trouble Shooting" Checklist for Higher-Educational Settings is one 
of two instruments designed to measure an organization's potential for success- 
fully adopting and implementing educational innovations. • In particular, the 
instrument focuses on a department's organizations^! climate, staff, communi- 
cation patterns, innovative experience and students. Another form of the TSC 
focuses on similar characteristics of elementary and secondary schools (The 
"Trouble Shooting" Checklist for School-Based Settings). 

\ .■ - . 

ReseaTch in the area of innovation adoption has primarily focused on three 
niaior areas: the adoption-diffusion process; characteristics of innovations 
which make an innovation easily adopted; and, identifying characteristics of 
organizations in terms of "innovativeness . " This instrument primarily takes 
advantage of literature in the last category and attempts to tie it in with the 

"real-Wi^rld experience of change agents. By. identifying institutions which are 
not in a sufficient state of readiness to adopt innovations, the TSC can save 
potentially wasted, time, effort and money. "As the development of new educa- ^ 
tioaal products and. processes is becoming increasingly centralized, the number 
of programs ready for adoption is rapidly increasing. Many institutions are 

.seeking g.nints to adopt and implement these innovations witho.ut being 
sufficiently prepared to use tl^e materials as the developer intended. The TSC 
should be useful for both change agents and in-hous*e personnel in identifying 
strengths and weaknesses of an organization in relation^tp the adoption of 
innovations . ' 

The first form of the TSQ (Manning, 1973) was innovation-specific and 
focused onlv on higher-educational institutions. Subsequently, there have been 
three experimental, innovation-free forms of both the higher-educational and 
school-based TSC*s. 120 institutions have been rated on the various experi- 
mental forms of the TSC. These experimental, forms have resulted in the present 
two final forms. Since these forms have not been used in experimental or field 
studies the author invites, others to use the TSC in research and development 
activities. 

The development of thes6 instruments was funded through the National 
Institute of Education, the Procedures for Adopting Educational Innovations/ 
CBAM Froject at the Research and Development Center for Teacher Education, 
The University of Texas, Austin, Texas. In particular, I want to thank 
Gene Hall, the project director, for. his support, and Ron Fox, Archie George, 
Sue Loucks, Beulah Newlove, Eddie Parker , Bill^ Rutherford, and all of the 
individuals who participated in the data coll^lltion. I want to especially 
thank Donna Buntain who has not only contributed her skills and expertise 
throughout the entire developmental period, but has provided me with invaluable 
encouragement and support. 



Brad A. Manning 
June, 1976 
Austin, Texas . 
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General Description of the TSO ' 

The "Trouble Shooting" Checklists (TSC's) have been developed to assist 
educational change agents, and faculty and administrators concerned with change, 
•in their assessment of organizational variables predictive of an institution's 
potential fot successfully adopting innovations. Two final forms 'of the TSC 
are now existant: the TSC for school-based settings (K - 12th grades), and 
the TSC for higher-educational settings (university and college level). This, 
manual focuses on the higher-edixcational TSC, while^ another xrianual- is available 
for the school-based TSC (see Manning, 1976). ' ^ - 

The TSC is a diagnostic and predictive instrument designed to aid users in 
estimating the effects of particular variables on the adoption/diffusion pro- 
cess. That is, the TSC provides users with a means of systematically organizing 
descriptive information in a predictive way. Because the TSC is broken into ■ 
five s*Siles, a profile emerges, indicating particular strengths and weaknesses 
within a department (with respect to the adoption process). 

Scales ~ 

- The TSC consists of 100 Likett-^type items which can be broken xnto the 
following five scales: ^ 

r. Organizational Climate: This scale focuses on the organizational 

climate' wtthin tTi^de'partme^ In particular, this scal-e is- concern- 
ed with the power system within the department, the kind of behaviors 
that are reinforced, organizational values and norms, and "openness" 
of the department. . 

.II. Organizational Staff: This scale focuses on personality and l^ader- 

$hip characteristics of faculty and administrators as they are related 
to the successful adoption of innovations. In particular, this cate- 
gory is concerned- with interactions between faculty irTembers, between 
administrators, and between faculty and administrators. In addition, 
. this category seeks to'' identify attitudes and interests of the faculty 
and administration as they are related to innovation. 

III. Communications: This scale focuses on the communication process asso- 
ciated with successful .adoption and implementation , of ^innovations. 



Particular femphasis is placed on the degree to which information exchange 
is superficial, restricted, or productive. In addition, this category 
attempts to define the quality of communication between change agents 
(both internal and external) and the department.* . \ \ 

IV. : Innovative Expejrience : This scale focuses on the expertise and degree 
• of. sophistication that an institution has had with the adoption and im- 
plementation of innovations. Special concern is placed on such factors 
as the degrees of awareness of J)asic information .about innovations, and 
indications from the faculty that they have some idea how to integrate 
' an innovation into their teaching.<^ 

V. \ Students: This scale focuses on* characteristics of students which^can'^ 
-'.affect the' adoption- implementation process. In assessing these charac- 
'te1ristic3, students' attiutties towards, the faculty, and their course 
work^^re considered to be crucial, as well as student enthusiasm, stu- 
-^Vdent iriteraction with* peers and faculty, and student individuality. - - 

Theoretical Framework and Origin 

The "Trouble Shooting" Checklists- have been developed In con*j unction with a 
project which has as its theoretical framework the Concerns-Based Adoption Model 
(Hall, 1974; Hall, Wallace & Dossett, 1973). \ In brief, the Concerns-Based 
Adoption >todel (CBA14) focuses on an individual's Stages of Concern about, and 
Levels of Use of, an innovation. The ordering of concerns and use are postulated 
to be progressively more sophisticated throughout the adoption-implementation 
prpcess. The relationship of the TSC to the Concerns-Based Adoption Model is 
based on the assumption that in order for Stages of Concern and Levels of Use 
to develop progressively through thie adoption-implementation process, an insti- • 
tut ion must meet certain conditions and be in an appropriate state of readiness. 



The TSC is designed to aid in predicting and diagnosing an institution's state of 
readiness. 



. Uses and Procedures of Administration 

~" ■■ ?■■■ 

Uses of the TSC . The TSC has several majot purposes. First,. the TSC in- 
tends to provide an overall norm-referenced, predictive score which estimates 
the likelihood of. a department to successfully adopt and implement an innovatiorf. 



Secondly, the TSC intends to provide a five scale diagnostic* profile which fo- 
cuses on the^strengths and weaknesses of a department's environment iA relation 
to the adoption and implementat-i^r^^^ji^novations . 

Within the context of being both a predictive and diagnostic instrument, 
the TSC may be used by several population groups for several different purposes: 

1. Change agents (either internal or external) may use the TSC as a pre- 
dictive tool to. identify -institutions which would be most likely to 
adopt an innovation successfully. Such use of the T$C could assist* / 

. a change agent or funding agency in determining whether commitments 
of time and money would be worthwhile in particular departmental set- 
tings, 

2. Change" agents (either internal or external) may use thef^SC as a 
diagnostic tool to obtain information that would help in planning in- 
tervention strategies appropriate to a particular department. For 
example, if a department scored particuarly lo\i on. one scale, a change 
agent could plan interventions which would strengthen, the department's 
particular weakness (e.g., establish new communication networks; etc.). 

■ / ' " 

' 3. Adminstrators and faculty interested in change may use the TSC to iden- 
tify problem'areas within their department.; If a department is con- ' 
sidering the adoption of a new program, members cou-ld use the TSC 
to self-evaluate the department and identify strengths and weaknesses 

. ; which would effect the adoption process. 

'4. Several members of a department may complete the TSC in order to iden- 
tify differences in*their. perceptions of 'the department. Such evalua- 
tion could -be catalytic to discussions of problem areas and differences 
of perception within the department. 

5, ' Educators may use the TSC as an instructional tool to teach change 

agents or students interested in change, to identify key organizational 
variables which affect the adoption process. 

6. Research organizations may use the TSC to identify highly innovative 
or noninnovative departments for pilot testing of new programs, in 
order to measure the effects of the program in varied settings. ■ 

Limitations on use . Users should keep in mind that the TSC is a new in- 
strument. While the TSC does have reliability, some indications of validity and 
norms based on a limited sample, the TSC has not been used extensively in either 
experimental or field studies. Therefore, the developer cautions users not to 
rely solely on TSC scores for decision making. 



Administration of the TSC . The TSC is easily completed and hand scored 



(see scoring section). The respondent is sirmply required to respond 'to each 

descriptive statement on a 1 5 scale (ranging from "very typical" to "very 

■t, ■ ■ ' . 

atypical"). In addition, the respondent may use either an "NA"^r "?" to in- 
dic^te if an item is N(Dt Applicable to the particular institution (NA) , or 
if^'' s/he ' does not»have the necessary information (?). If ^ respondent uses a 
^.^?" for a response, s/he shduld try ta» obtain the needed information before 
scoring the instrument. Time required to Complete the instrument, af^er an 
individual has f amiliariz^ed him/herself with an institition, is estimated. tOi 
be 2iD - 30 minutes Scoring is estimated to take 15 minutes. 

•-4k'. 

Members of an organization should be able to complete most items based on 
their knowledge of the institution. External change ag(rfnts should f itst study 
the instrument in order to become completely familiar with the informat^ion* re- 
quited to complete the form, and then spend a few days meeting with and inter . 
. viewing various members of th§ organization in order to obtain the necessary 
information. The author urges^that interviews be conducted with a variety of 
people, both receptive and unreceptive, of varying influence in tbe organization, 
in order to obtain the- i^ost complete impression of the institution. In such 
interviews, students should not be neglected. ,^They can be a valuable source of 



information. 
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^ Technical Development of the TSC . ./ . ^/ 

* ■« ' ■ .■ , • 

Origin of the TSC ^ ' ' \ ' * ' . 

The early development of the TSC or Higher-Educational Settings has-been 
documented- in detail in Manning (1973rand two AERA papers (Manning, 1974; Man- 
ning, 1975). In brief, the TSC was first developed as a survey- form' (TSQ - 
"Trouble Shooting'* Questionnaire), which was used to collect the information . 
upon which the present TSC is based. "The TSQ was a twehl^-nine page op*en-ended: 
questionnaire which was completed by six change agents. The questionnaire was, 

c ■ ■ • ■ ■ ^ ^ '■" _ ■ - , 

innovation-sgecif ic, focusing only on institutions adopting modules and personal 
assessment feedback ' (PAF) . In addition, ' the questionnaire was broken into ideal, 
marj^inal and unacceptable question areas. The questionnaire was given to ex- • 
perienced change agents. wht5 Wrote sentence and paragraph responses to the ques-i 
tiohs. These responses wer^e then synthesized to make them as succinct as pos-. 
sible without-losing descriptive information. Items were also sorted into o. 
natural groupings within each .question area. The initial draft of the TSC 
(Checklist) form, thus, consisted of two innovat iot^ii^ecif ic instruments, the 
TSC-A and TSC-B (one instrument for institutions adopting modules and one. for 
tnst it ut.ions .adopting PAP). Each instrument had five information a,reas r 
correspond:lne to the natural gro.upings which had emerged. IlS addition, items 
were assigned score values of 2, 1, and 0 for items which h^rd been generated 
from the ideal, marginal and unacceptsible question areas on-the tSQ. 

The first ^innovation-free form of the TSC for higher-educational' settings 
was based on the TSC-A, TSC-B, and a literature search. Tte first step was to 
eliminate or modify all items in the- TSC-A and TSC-B which specifically referred 
to modules, counseling -or assessment batteries, as well as it^fems which had any 
innovation-specific referrent. The remaining items were then pooled to ^form the 
new instrument. In addition, some items were added based on the literature 

10 



search. This first experimental, inrK|vati6n-free fferm of the TSC*£or higher-.. - '^^ 

educational institutions consisted of 495 items- classified as describing inno- 

■ .* i ■ . 

yative, marginal or unacceptable organizational characteristics '(165 items of ^ ' 
each kind). Tttese 495 items were organized into I6^subscales in 5 major scales^ 
and occupied 39 pages. All subscales were forced-choice requiring that 1/3 of * 
the items in any one scale be selected. . ' . . 

L This first experimental, ^^nnovation-free form of the TSC for higher-educa- < 
tional settings was distributed tc/ a small, natior-wide sample of change agents 

who were asked to complete th^ TSC, and to critique and^eoiAm^t upon it ifl detail. 

• . * • ■ / - . ' r - 
In such a way, it was possible to obtain de^t-ailed responses from a representa- 
tive sample o£ would-be users in addition to obtaining data necessary for the first 

item analysis. A synthesis of the critiques w^s compiled and remedial actions 

f • ■ ' . ■ . 

outlined. ' ^ ' ^ ■ . ' \ 

/ .. • . \ . ■■■ ^ , 

Based on this sampling procedure, it was decided to narrow the instrument 

to 100 items formatted in a 5-point Linker t- type scaXe.- Consequently, all marginal 

■ . ■ . _ • * • . ■ . ■ * . ■ ' 

items were removed, except those whi(^h correlated highly in the-- final analysis 

after having been re-classif ied^as ideal or unacceptable based^on one of the 
analyses . . . ' ^ ' . 

Initial Item Analysis. 

^^'^ - • ' ' ' - . V . 

A series of item analyses were completed en this ' first 495* item experimental 
form of the higher-educational, innovation-free TSC- Information , - 
on these analyses is available in Manning (1975). .v ' - . * 

150 items were selected on the basis of these analyse.s, and comprised the 

second experimental form of the higher-educational TSC, These psychometrically_ 

' -. " / • • / ' • - ■ ■ ■ , --v . ,. 

sound items were also examined in terms of the detailed comipents made by change 

agents who had completed the checklist, ^ome items were then rewritfien for 



greater clarity. ^ J[ 



In sura, the siecond experimental form of the higher-educational TSC con'sisted 

. . ■ # • . * 

of the '''highest 'correlating 150 items d^awn from an original pool of 495 inno- • 

vat ion-free items (which, in turn, were based on the TSC-A and TSC-B) • Because 

' ■ ^' \ ' * ■ . 

the format of the TSC was changed Vo Likert-type scaling, the 150 "items were " . • 

* * ' '• • ' '. • 

drawn primarily from ideally and unacceptably classified items (only a few 

- ' ■' . ■ ■ ■ i 

marinally classified items were retained). These 150 items could be broken 

into the following five scales (subscales were .^eliminated since the number of 

items was reduced): organizational climate; organizational staff; communications; . 

innovative experience; and, students. 

This second experime^ntal form of the; higher-educational TSC was submitted to " 
v ■ ■ \ ' • 

organizational development specialists iri\ order to obtain. suggestions>! for modi- 

■ . . ^ • \ ■ ■ ' . . 

fication of individual items. After final "Revisions were completed, the third 

experimental form of the higher-educational TSC; emerged. Because of the re-writing 

■•»■•' , • * ^ ■ 

•o^ items and the change in format, the higher-educational TSC underwent another 

series of iteA analyses before beings fihalized into its present form. The 

"second series of analyses were us^d to narrow the number bf it^iris down to 100. 

The analyses which were conducted are described bel6w. 

Final, Item AnaS^sis * ' « . . ' 

Fafty-sevenXhig^he^r-educational/personnel were asked to anonymously complete 
the^^ird-'experimental f drm af tlie higher-educational* TSC. Only 30 TSC^^p/ere 
returneH' in time to be inclu'Sed in the data base for the final item ajialyais. 

The first analysis f ocus'ed on "the following question: do items assigned to 
one of the two groups of items (items describirig /innovative institutions, and. 
items describing noninnovative institutions) belong..with their respective groups? 
This analysis consisted o^ correlations betweeTi each item and the total score 
•'for each group of items (gee* Table 1) . The ..alpha coefficients for the two groups 
of items were as follows : items classified a^' innovative -a = .96; items classified 



as noninnovative a = .96; and, total a = .98. 

The second analysis, focused on whether or not items were internally con- 
sistent within each of the five scales. This analysis 'consisted of correlations 
between each item and each of the total scale scores (see Table 2). One result 
of this analysis wa3 that the scales demonstrated a lack of independence. These 
results indicate that institutions which rate highly in one scale are likely to 
rate highly in other scales as well. 

*The final analysis, like the second analysis, focused on^whether or not 
items were infernally consiste;nt within each scale and: within t^e entire instru- 
ment (see Tabl^ 3). This analysis resulted in correlations of eac^item with ' 
the total score of the instrument and with its totals scale- scores. The alpha 
coefficients for the^fxve- scales and total instrument^ are presented in the 
table below. 



Scale number 


Items in scale 


Alpha 


1 


30 


.90 


2 


30 


.91 


3 


30 ; 


.94 


4 


. 30 


'.93 


■ ■ . 5 


30 


.90 


. Total 


. . 150 


.98 









Item Selection . ' ' 

The analyses described above provided Jthe basis for the selection-, of the 
final 10.0 TSC items. The /selection procedure required 'the compilation of a 
^detailed sumiiiary of the analyses co^ftaining each item followed by correlations 
on each analysis. In 'such'^ way, the results of all analyses could be reviewed 
at once, in order to determine whicti items correlated highest across the analys( 
In addition, items were examined in terms of content. When several individuals 
failed to respond to a particular item, tjie item was examined to determine if 



it required information that was difficult to obtain, or whether the item re- 
quired information which was easily obtainable , -but unknown to the respondent 
In addition, particularly in borderline cases, the content of items was also 
considered in order to insure that a full range of variables would be repre- 
sented within each scale. 

The final 100 items wore then re-analyzed for new alpha coefficients in 
order to determine the reliability coefficients* for the final five scales and 
total scores. The alphas for the final 100 TSC items are presented in the 
table below: ' 



Scale Number 


Items in scale 


■ Alpha 


1 


19 


.89 


2 


22 ' 


.93 


3 . 


19 


.94 


4 


22 


.96 


5 


18 


• .91 


' Total 


100 


.98 



Format and Scoring 

The TSC consists of 100 descriptive statements. These statements are ran- 
domly arranged, and can be broken into the following five scales: 

Scale I: Organizational Climate 

(Items in this scale describe the work climate and organizational 
structures of both the department and the institution as a whole.) 

Scale II : Organization Staff 

(Items in this scale describe personality and leadership styles 
of faculty and administrators within the department.) 

Scale III: Communication 

(Items in this scale describe communications both within the 
department and witTiin the institution as a whole.) 

Scale IV: Innovative Experience 

(Items in this 'scale describe a department's experience with 
innovations and attitudes toward innovation.) 

Scaie'^V: Students ^ . 

(Items in this scale describe student behavior, attitudes and 
.demographic characteristics.) 

Each item is to be rated on a five point scale ranging from "very typical" 

». . 

to "very atypical." A "?" may be used when one doesn't have enough information 
to respond to an item, and the symbol "NA" may be used when a. statement is not 
applicable to a particular department. The instrument provides five scale 
scores in addition to a total score. 

Scoring . ^ 

Scoring of the TSC can be done 'by hand and recfiiires approxinjately 15 minutes. 
As explained in detail below, all items which describe noninnovative ^organiza- 
tional characteris^tics must be reverse keyed -before the scores are summed. Scores 
are. then added for each scale and for the total - instrument . Those respondents 

.wha have chosea. to use the symbols "NA" and "?" should refer to the Score adjust- 
ment formula which provides a^ foritiula for equalizing' the scores of TSC's in c . « 

'which these symbols were used, with the scores ^of TSC's in which these symbols 



• 11 

were not used. 

Reverse key- scoring . The item numbers listed beloW are reverse keyed, 

•and should have their rating values adjusted in the following manner: 

Reverse ^ 
Item score 
. ' response value . • 

5 . . - 1 
4=2 

3 = 3 (reverse keyed items rated 3 should 

not be changed) • 

2 - A .: . . 

• r 1 . 5 

? = 0 

... - c....„,-......NA 0 , . 

' For example, if ycu have marked one of the following items a "1," it should be 
changed to a "5" for scoritig purposes; if you have marked one of the following 
items a "4,^* if should be changed to a "2" for scoring purposes. The following 
items should be reverse keyed: ^ . 



1 


13 


30 


^3 , 


55 


67 . 


81 


96 


3 


\ 18 


31 


44 ^ 


57 


, 68 


85 


97 


4 ' 


22 


32 


46 


■ 60 


73 


86. 


100 


5 


24 


33 


47 


61 


75 


87 




.6 


27 


34 


48 


64 


.76 


91 




7 


28 


36 


51 


65 


78 


9^ 




11 


.. . 29- . 


37 


52 ' ■ 


66 


80 


93 





Scales . In order to derive each scale score, add the^ ratings for -the 
^xaHfiaciilve Item num bers listed below (the_symbQlsJ'?'' and "NA'* should be assign- 



ed 0 score values).: 
Scale I: 



Scale IL: 



12 


30 


52 


70 


84 


•-- 15 


" 33 


'58 


■ '78 


89 


23 


36' 


62 


79 


.92 


27 


51 


66 


80 




,4 


32 


45 


59 


. 73 


5 


35 


47 


60 


95 


8 


37.-' - 


48 


61 




28 


40 


53 


63 




29 


'44 


54 


71 
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Scale 


III: 


7 


38 


57 


89 








1 -7 
1/ 


39 


64 


o3 


n "7 

9/ 






OA 

zu 


42 


^ c 
03 




1 (\(\ 

100 










59 


Q"7 

o/ 




Scale 


IV: 


3 




41 


74 


98 






/: 
O 


19 


4o 




on 

yy 






9 


o o 


en 


/o 








11 


24 


^ o 
DO 


93 








1 o 

13 


34 


72 


94 




Scale 


V: 


1 


18 


49 


77 


90 






2 


21 


55 


81 


91 






10 


25 


56 


86 








14 


26 


67 


88 





Deriving total score . After the item ratings have been 'changed as^ des- 
cribed, above, ^ add all ratings to the left of items ^or the total score. 

Score adjustment formula . If you have chosen^ to respond to items with 
the symbols or "NA, " it is necessary to use the following score adjustment 
formula. The score adjustment formula equalizes the scores of TSC's in which 
these symbols have been used, and the scores of TSC's in which these symbols 
have not been used. This formula assumes that the items receiving a ntimerical 

response are representative of the entire- scale content- All items rated "NA" 

- 

receive a scpre yaiue of\0 in this formula. ' . o 



computed score 

for scale ; . 

^ — ;:: — : X Number of items in scale 

numbe r of items not ~ . 

marked with a or <r 

a "NA" on sqale ' . 

Fpr example, in order to score Scale I, first reverse key^ items as explained 

above. After reverse keying the items ^ add up the total score (giving and 

"NA" scoring vaiue of ^) . If an individual has two "?*s," twb "NA's'' (all 

of which count as ^ towards the total score) , five "2 's,"^- and ten "3's," the 

formula' would be completed as follows: 
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Actual 

computed score 40 

for scale ^ X 19 (Number of ^ X 19 = 51 



Number of items not / ' items in 

marked with or 15 scale) 

**NA" on scale 

..The" score for Scale I is therefore 51. 

Likewise V t4\e score adj^istment f ormala -far^4:he— t^tal-^G4«i~i^— as— fo-llows; 

Actiial computed . 
scor^ for entire TSC X Number of items in TSC 
' . Number of items not (100) 
^.marked with a or 

"NA" on entire TSC . ^. . 

.For example, if an individual rates an institution using five "?'s," f-ive *'NA's 
rating five items with a value of "1," ten items with a value of "2, twenty- 
five items with a value of "3," twentyrfive items with a value of "4,''\ and 
twenty-five items with a value of /'5," the formula would be computed as follows 
(reverse keying the items) : ^ . 



325 
90 



100 = . 361 
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Indications of Validity 



Due to limited resources at this time, a full scale study of validity has 
hot been possible. However, some indications of validity are available. During 
the item analysis study , respondents were asked to complete a subjective rating 
form {see Appendix A) on the. same institution which- was rated on the TSC, On 
this form, respondents were asked to indicate their subj.ective assessment of the 
institution's potential for successfully adopting educational innovations. On 
a scale of 1 - 4 (1 indicating no potential for innovation, and 4 indicating 
excellent- patentaal for- innovation')-^ -respondents- were ^sked to provide-^anrbveirall 
rating and 5 scale ratings. Each resp6ndent's subjective ratings were then cor- 
related with his/her TSC scores. These relationships were analyzed using a multi 
trait-iiHiltimethod matrix (-Campbell & Fiske, 1959), and examined in terms of con- 
vergent and discriminant validity in order to explore the possiblity of any 
indications of v^i^idity, - . . . .. 

In order to establish convergent validity, there must be a significant cor- 
relation between two diffetent measures of thev'3ame trait. Discriminant validity 
requires that: ' . ' ^ 

The correlations between ^two different methods measuring the 
same trait exceed^ *(a) the correlations obtained between that 
trait and any other trait not having method in common and 
(b) the' correlations between different traits which happen 
to employ the same method. Variance among test scores can 
be due to method and/or trait factors. The multitrait-multi- 
method matrix presents all the intercorrelat ions which result 
when selected traits are measured by two or more method? 
(Borich & Bauman, 1972, p. 1031). 

Because the subjective ratings were made by the same" person who completed 

the instrument, this stifdy does not qualify as a true validity study. However, 

since this comparison is the only data available upon which to base indications • 

-Of _,.validity , the data were analyzed as in' a validity study, . ' ' 

; Examining the four quadrant multitrait-multimethod matrix for the corre- 
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relations between TSC ratings and subjective ratings (see. Table 4), there is 
evidence of convergent validity. As evidenced in Table 4, all of these corre- 
lations are significantly different from zero. These correlations, starting with 
Scale 1, through Total Score, are as follows: ./76, ,696, ,757, ,581, ,357, and 

--^SS^~ -Sueh--^mrrel^lnOTis~-a of convergent validity. 



The first method of establishing discriminant validity is to determine if. 
the values in the validity diagonal (see Table 4) are higher than the values in 
the corresponding rows and columns of the adjacent correlation triangles (hetero-" 
trait-heteromethpd triangles)^. For example , j7hen_,77^^6^^^^^ 

TSC Scale 1) is compared with the correlations across the quadrant^ (first row) 
and down the quadrant ([first column), . 776 iS found to be higgler than h out 
of the 10 correlations and approximately the same as one of the correlations. 
The rest of the ^correlations arenas follows: ,.696 is higher than 3 correlations; 
.757 is higher than 5 correlations; .581, is higher than 6 correlations; .357 
is higher than 9 correlations; and ;^55 l^bjective overall rating with total 
TSC score) ds higher than all^ 10 of its associated rOTr^and^-Xolumn correlations. 
Using this procedure, there are indications of diseriminant validity on],y~tor-~__ ^ 
Scales 5 and the Total instfumen^,. 

A :secona procedure used to ' establish discriminant validity requires thaf 
■ the values in the validity diagonal be:> higher than the triangles in the first 
and fourth quadrants (heterotrait-monomethod triangles). These triangles re_pre- 
sent the common influence of the same method on the 5 scales and total scores. / 
In other words, this second criterion requires that the trait variance should 
be larger than the method variances . As can be seen by examining the^ table, the 
first correlation, .776 (reading down the validity diagnoal) is higher than all 
15 correlations 'in the heterotrait-fnonomethod triangle directly above the validity 
diagonal. This same coefficient (.776) compared to the heterotrait-monomethod 



20 



16 ■ . ' ' • . ' • ' . ' 

triangle in the fourth quadrant (subjective rating correlated with subjective 
rating) is higher than ^5 out of 15 correlations. The remainder of the corre- 
lations are as follows: .696 is higher than 12 correlations in the first qua- 
drant, and higher than_^_oorxBlatJjms--i4i^he-€^u^^ 



than all 15 correlations in the first quadrant and higher than 5 correlations 

in the fourth quadrant; .581 is higher than 8 correlations in the first quadrant 

and higher, than 5 correlations in the fourth quadrant; .357 is higher than 7 

0 

correlations in the first quadrant and lower than all correlations in the fourth 



qria^dfattX ; "and, the last correlation in the validity diagonal, .855, is higher 

than all 15 correlations in the first quadrant triangle and higher than 6 out ^ 
of 15 correlations in the fourth quadrant triangle. As evidenced in the table, 
all correlations in the fourth quadrant (subjective with subjective) except for 
Scale 5, were higher than the correlations in the validity diagonal of the third 
quadrants Thus, using this method, there is not evidence of discriminant validity. 
However, the high intercorrelations among the, subjective jscales may indicate 
a unitary factor (see the discussion section). 

A third criterion for the establishment of discrimiijant validity requires 
that the same pattern appear- in ^11 the trait-met|i6d triangles discussed above. 
As illustrated below, the four i>eterotrait (both monom'ethod and heferomethod) ' - ^ 
"^X^ triangles do have similar patterns of correlations (high and low are, of course, 
relative to each triangle, since the fogus is on pattern aij(i'not size of the . 
. correla'tions) . The table below describes this pattern: 





1 


2 


3 


4 


5 


K 


•high 










3 


\ high 


high 








4 




low 


low 






^ 5 


lovK 


low 


low 


low 


Total 


high \ 


high 


high 


high 


low 



^hjrS-m^thed--Q'f- ^eLei U iiulug ^x[±sr^^ ^valiaity", it can be concluded that 
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an indication of discriminant validity doeis exist. 

,' 0 ■ 

Discussion of Validity • • 

- In anal yztrrg t he re sults of~this. study, it should be emphasized that the ., 
above discussion is not a validity study in the usual sense. Instead, ^it^ is 
•a study of the relationship between clinical subjective ratings of an insti- 
tution's innovative potential (with respect to 5 areas of an institution's 
environment plys an overall rating) and the corresponding TSC ratings: In each 
of the 31 institutions included in the analysis, both sets bf ratings* were made 
by the same individual. A true validity study would have, of course, contained 
an independent set of subjective ratings of the same institution made by a dif- 
ferent group o£ judges. In other words, in this correlational analysis the 
approach, of necessity, was limited to ];iaving the same raters use a different 
means .to. rate the same institution. In a true validity study, the instruments 
w5uld not only be different, but the raters would be different. 

Only one method of determining discriminant validity ' indicates validity 
(see method 3). The dat^ .suggests that the subjective scales do not discriminate 
in measuring all the factors of the TSC scales and instead, measure a single 
• factor. The "six ratings may be more reflective of a global att;itude, than of 
specific judgements of different aspects of an organizational environmeut due 
to the brevity of the subjective ratings. The possibility^ of a global attitude 

is buttressed by the fact that all of the correlations in 'the ^subjective with 

• " ' \ • ■- 

, subjective quadrant (except 5) correlate, highly (see ^the second ihethod for 

determining discriminant validity). In ^some cases, specific subjective ratings 

correlated higher with other TSC i'scales than with their own scale (e.g., TSC 

Scale 1 correlated higher with subjective Scales Z and 3 than with its own ^ 

scale). 

Li^e iesu ltrs^"~t1ie^an^^ subjective ratings with TSC scores 

22 
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are inconclusive. The similar patterns which emerged in the 3rd method of 
detepnining discriminant validity provide the only indications of discriminant* 
validity. The fact that similar patterns emerge- provides evidence against 
the hypothesis' of a unitary factor. However, results of other methods of estab- 

lishing^ discriminant validity as well as the higher intercorrelations of the 

■ . ' ^ ■ 

fourth quadrant, suggest the possibility of a unitary factor, A true validity 
study must be conducted to determine whether a unitary factor exists, or whether 

J ■ 

the instrument demonstrates discriminant validity. The author invites others 

to complete such a study or contribute inf ormat iorj^^owards sutth a study. 

Finally, the student scale correlations (Scale 5) should be Interpreted 

with caution. Many of tl)(e respondents reported that, they were insufficiently 

familiar with, the students to be confident alDout their ratings on this scale, 

/: • • 

Content validity is evidenced in the development section of this manual, 

as well as in two other papers (Manning, 1974; Manning, 1.975), The.|.nstru- 

ment is based on both research literature and change agent responses to questions 

focused on the information, areas ""contained in this instrument- In addition, . 

other professional researt:Kers on the^ sponsoring project have offered their 

suggestions -and critiques thrioughout the developmental process. Finally, 

organizational devlopment specialists, contributed suggestions for revisions. 



Table 4 



COliRELATIONS BETWEEN TSC RATINGS AND SUBJECTIVE RATINGS 

(N-= 31) , ^ 
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The norms are based on a ^roup of '32 iMividualis wlip^^^ul^ users 
of the TSC. They all worked in higher-educational settings, In roles of teachers, 
administrators, or change agents, and had an interest in the development of an 

0 ■ 

instrument predictive of an organization's change potential. They filled out 
• the instrument on an institution with which they were familiar , with the under- 
standing that they would not Have to identify themselves or the institution which 
they rated. A table of Percentile Equivalents for Higher Educational TSC scores 
is presented in Tablje 5. This table gives percentile ranks and their corre- 
sponding raw score values for all five scale and total scores. In addition, ■ 

... 

means, medians, standard deviations and standard errors of the means are pre- 
* . ■ ■ 

sented for ^each of the five scales and total score. ^ . ,^ 

Of course, notms based oii^^n N of'^SZ-iife^^ limited* value , /but they represent^ 
a" beeinnine. The.^lns^ument^appears to be^TnTerhally consi sten t , and some^>f{aica£ions 
of validity have bee^ presented. Furth^ ysof the instrument seems -to be 
war«inted. The developer invites any .institutions using this instrument to' share 

their data, so that more extensive norms can later be published/ It is, pf ^ 

' . . . ■ , ' ^ - 

course, also hope"d that groups of institutions will generate, their own norms. 



Table 5 . . ' 

PERCENTILE EQUIVALENTS . 
^FOR HIGHER EDUCATIONAL TSC SCORES 
■ ^ ; (n' ^, 32) ' ' 



Percetitile 
Rarik 

100 . 00 

96.9- 

9 3., 8 

9d".6 

87.5 

84.4 

81.3 

78.1 . 
°75.,0 

V .9 . 

68.8 . 
65 . 6 

*62.5 
59.4 
5^.3. 
53.1 . 
50.0 

46.9 ■ 
43.8' 
40.6, 
37 .,5'' 

• 34 4 
31.3 
28.1 
25.0 
21.9^ ■ 
, 18 . 8 
'l5.6 
' .■ 12. ? 
■ 9.4 
6.3 ■ 
3.1 
0.0 



85. QO 
83.00 
82.50 
82.00 
78.00 
77.00' 
76.00^ 
75.00 
74.00 
73.00 
71.00 
70.00 
69. OQ 
*67.0'0 
65.00 
64. '66 

64 . 00 
63.25 
6.2 . 50 ■ 
61.75 
61.00 
68.00 
53.00 
'46.00 
45.50 
45,00 
•44.50 
,44.00- 
' 4?, 34 
r42.67 
42. '00 
41.00 



Scale Score Values 



II 



III 



IV 



101.00 

■ 98.00 
'97.00 

96.50 
96.00 
' 9.4 . 50 
93 . 00 
52.50 
92.00 
91.50 
91.00 . 

■ 90.00 
• 89.00 

86.00 
. 83.00 
n82.00 
81.00 
80.00 
79.'00 
78.00 

* CS-7.00 

73.00 
72.00 
69 . OQ 
. 63.00 

• 59-. 00 
\ 58.66- 
! 58.33, 
•58.00 
52.00 
50.00 
47.00 
46.00 



i8'5 . 00 
82.00 

■ 81 . 00 
80.50 
80.00 
?8.00 
77.50 

.77.00 

76.00 

74.00 

73.50 

73.00" 

72.50 

72^00 

. 71.A0- 
69.00 
68 . 34 

' 67o.67 
67.00 
66.00 
64.00 
63.00 

'54.00 
49.00 

. 48.00, 
47.00 
44.00 
42.00 
38.00 

. 36 '.00 
35.00 
34 . 00 
'33.00 



lOO/OO 
99.00 
96.00 
95.00- 
92.50 
90.00- 

89;oo 

'85 . 00 
84. '00 
83-. 3b 
82.00 

• -SLOO" 
79.00 
78.00 
7 7.00 
76.00 

' 74.50 
'73.00 
70.00 
68.00 
66.00 

' 62.00' 

■ 61.00 
59.00.' 
57.00 
53.00 
49.50 
46.^00 

■ 44.00 
42.00 
40.00 

■ 36.00 
35.00 



82.00- 
78.00 
75.00 
74.50 
74.00 
73..t)0 
72, 00. ■ 
71^50 
•71.00 
70.^0 
68.50 
67.00 
66.00 
65.00^- 
64., 00'' 
63.' Op 
62.00 
58.00 
54.00 
52.00 
51.00. 
49.50 
48.00 
47.00 
4 6.'. 00 
45.50 
45.00 
44'. 00 
42.00 
39.00 
27.00 
26.50 
26.00 



Total 



438 
437 
437 

- 427 
418 
417 
'410 
391 

- 390 
38^ 

•374 
373 
.372 
. 368 
367 
363 
360 
357 
355 

»sa347 

• 339 
324 
300 

^ . 274 
250 
248 
243 

' 234 
233 
224 
216 
208 
207 



.00 
.50 
. GO' 
..00! 
.'00 
. 00 
.00 
.00 
.60 
.00 
..00 
.00 
.00 
.00 
.^0 
.00 
.00 
.00 • 
.00 
.00 
.00 . 
.00 

;00 
.00 
J30 
.00 
.00 
.00 
.00 

.'00 

.00 
.00 
.00 ' 



Mean 



63.94 



79.28 



,64.06 



71.91 



57.94, 



340.44 



Median 



65.00 



82.00 



69.00 



76.00 



62.50 



36'3. 00 



StdT' Dev. 



13.55 



15.91 15.83 




18.46 



. Std'. Error' 
of Mean ' 



^.39 



•2.81 . ,2.80 



3.26 



17.20 : 

^ 

3.04 



73.20 



12,94 



26' 
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Subjective Rating Sheet 



Please assign a- fating of 1 - 4 for eac^i of the following categories with 
respect to the depar.tment's potential f&v adopting innovations: 

4:= excellent potential for- innovation 
3 - good potential for innovation 
2,= limited potential . for innovation 
Ir = virtually, no potential for innovation 

^ 1. Overall, rating of department. ' 

2\ Organizational Climate : This category focuses on the organizational 

climate of the department. Particular ^cQuaideratlons^ should include 
the power system within the organization; the kinds of behaviors that 
are reinforced; organizational values and norms; and "openness" of the 
organization/ 

3. Organizational Staff : This category focuses on personality anh leader- 

^ ship characteristics of faculty and administrators as theiy are related 
. " to the successful adoption of innovatfions . In particular , this cate- 

' gory is concerned with interaction between faculty ijembers, between 

administrators, and between faculty and administrators. In addition, 
this category seeks to identify attitudes , and interests of the faculty • 
and administration as they are related to innovation. 

4. Communications : This category focuses on the communication process 

^ associated with successful adoption and implementation of innovations. 

Particular /emphases should be placed on the degree to which information 
exchange is superficial, restricted, or productive. In addition, this 
category attempts to define the quality of ^-communication between change 
agents (both internal and external) and the department. 

5. Innovative Expedience :. This category focuses on. the experience and 

~ degree of sophistication that an institution has had with the adoption 
and implementation of innovations. Special concern should be placed 6n 
such factors as the degrees of awareness ^f basic information about 
innovations, and indications from the faculty that; they have some idea 
how to integrate an innovation into their teaching. 

6. Students : This category focuses ^^n characteristics of students >which 

can affect the adoption- implementation process. In assessing these 
characteristics, students* attitudes towards the faculty , and their _ 
course work, are considered to be crucial, as well as eiithusiasra, stu- 
dent interaction with peers an5 faculty, and student individuality. 
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Each item correlated with score of own scale (innovative/noninnovative) and with 

total score of instrument. 

Items classified as innovative Items classified as non- innovative 

Gorrfelated with total score of correlated with total score o§ '^all 

all itemsclassified as innovative items classified as non- innovative 



• Item 
Numbeir 


R (Scale) 


R fTotai) 


, 1 






■ 2 






3 


.6674 


.6099 


4 


.5817 


.6479 


5 .' 

6- 


.8X72 


.8203 


7 

S ' 


.5067 


.5462 


9 .. 


.6545' 


. .-7231 


10 


.509.7 


.4712 . 


11 






. 12 


s 




13 






14 


..7775 


. 7657 


15 






16 


.3416 


.2232 


'17 


■ .t 




18 






19 


. ■ .3711 . 


^3922. 


• 20 




21 ■ 
"22 


• * 




. 23 


.5943 


.6432 


24 


.5586 


.5399 ■ 


25' 


- .-5'748 


. 514^ 


26 


.5554 


.5239 


27 


.7083 


.6672 


28 • 






29 ■ 


^5249. 


.4771 


30 


.5731 


.5820 


^31 






'32 






. 33 


.6724 


.6636 


34 


.6 482 


.6855 


35 


.52.74 


.6029 


36 


.5634 


.6111 


37 






38 


.8186 


.8208 


39 






"40 






41 


• .7681 


.7936' 


42 


.6891 


.6715 


43 






.- 44 






45 


.3927 


.4137 


46 


.6317 


.5627 ^ 


47 


.7159 


. 7357 


48 






4'9 
50 




^ 



R (Scale) 



R (Total)** 



.7419 
,5204 



,7839 
, 5568 



.7248 
.5724 
.6744 

.3447 

.7190 
.5891 

.6068 
.6567 
.6350 



. 7997 



.7556. 
.6862 



.3639 

.7008 
.5808 



.4707 
.'3762 



^6^62- 
,5283 
.-.4271 



.6724 
.5127 



.:8044 
.5936 



.7402 
.5673 
,6137- 

,3808 

,6873 
,6273 

,6692 
,6436 
,6330 



7980 



.7142 
.6643 



.3895 

.6880 
.5122 



.5261 
.4103 



,6135 
,4945 
,4051 



ERIC 



ijts.-tabie''lScXudes information only on the final 100 items selected, 
fotal score was obtained by inverting responses to the noninnovative items. 
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TABLE 1 (cont.) , 

,• . ' . • *" 29 





Items classified 


as innovative 


Items classified 


as non-innovative 




correlated with 


total score. of . 


correlated with 


total score of all; 




' ail items classified as innovative 


items classified 


as non- innovative. 


Item 
Numbe? 


K v.bcaie^ 


K v.iotai^ 




• TJ ( TnFal 


. 51 • 


.5373 


. 5189 






52 


'.7855 


.7775 






53 


.3662 


.3886 






54 






.7365 


.7932 


55 


.4621 


.3472 . 






56 






.4132 ' 


.3442' 


57 






.4563 


■ .4705 


58 


.5089 


. 4588 






59 






'.7331 


.6471 


.60. 






.6531 


.6319 


61 




. - 


.6131 


.5513 


62 


.7160 


.7404 






63 


.7101 


.6789 






64 


.3942 


.3397 






65 






'.8188 


' .8319 


66 






.5295 


.5159 ■ 


67 


.5270 


.517.6 






. 68 






.5688 A 


.5245 


69 


.3462 


.2651 






70 


.4553 


.4271 






71 






.5388 


.5305 


72 . 


.5496 


.5469 






73 






. .5073' 


'.5345 


74 


.6"982' 


.6747 ^ 






75 






.6124 


.5524 


76 


... ;5660 ' 


.5480 






• 77 






.7868 


.7265 


78 






.8480 ■ 
.7881 


.8378 


79 


V 




.729^3 


80 






.5891 


.6271 


81 






. 3390 


.3651 


82 






.5072 


.4957 


83 • 


.5451 


.5662 






84 






.3615 


.2984 


85 


.7045 


.6866 






86 






.5968 


' ■ .5743 


87 


.8296 


. .8630 






88 • 






.8947 


;9170 


89 






.4802 


.5844 


'90 


.7581 


■.'7278 






91 


.5486 


.5138 






92 


.5588 


.4476 




* 


93 
* 94 


r 




_1 .7245 ' 

.4607 

.7932- 

.7477 


.6742 
.3986 
.7842 
.7754 








, 97 






.4280 


.4386 


98 ■ 






.4472 


.3987 


99 


.6841 


.6757 






100 


.6493 


.6167 
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30 TABLE 2* . ' 

' o •• fc^ach Item Correlated With Total Scale Scores for Five Scales. 



Scales 



\ 



I tem 


Scale 


1 


2 


' 3 


4 


5 '• 


•Number 


Number 










. 1 


1 


lib 


C "7 T 

5/7 


5y 0 


• 

/UZ 




2 


2 


551 


Aon 

429 




/(Jo 


0 

9 


3 


4 


517 


c / 0 

548 


c 0 c 

525 




000 • 
390 


4 


1 


710 ■ 


c z' r» 

569 


coo 

538 


£ T r\ 

670 


351 


' ^ ^5 


..3 


762.' 


776 


825 


699 


C ^ T 

567 


6 


3 


747 


840 


Ikl 


T C 0 

75 0 


44o 


1 


" 1 


652 


574 . 


494 


5l2 


10/ 
;34 


8 


4 


561 


649 


Iff 
466 


coo 

592 


0/0 

349 


9 


4 


755 


679 


TOO 

733 


oOo 


117 


10 


3 • 


.429 


501 


422 


f'tr/ 

45A 


266 


11 


3 


710 


667 


769 


^ / 0 
643 


/ "7 0 

472 


12 


1 


508 


554 


601 


tr 0 0 

523. 


00c 
295 


13 


4 


659 


.493 


610 


^00 
639 


0/11 
261 


14 


4 


707 


700 


658. 


0/0 
o42 


lie 
446 


15 


5 


318 


376 


0 0 T 

281 


0/0 
349 


0 0 0 * 
393 


16 


i 5 


105 


1.33 


171 


59 


638 


17 


3 


741 


630 


670 


760 


1/0 
14o 


18 


2 


548 


634 


'574 


y 0 0 

49o 


C C T 

557 


19 


5 


296 


387 


0 ort 

330 


149 


. 691 


20 


2 


.^^545 


,645 


598 ^' 


591 


616 


21 


3 


"^650 


597 


728 


532 


*o 0 / 

304 


22 


2 

3'^ * 


610 


670 


591 


589 


0 T C " 

315 


23 


526 


706 


689 


ceo 

550 


0 c c 

355 


24 


2 


602 


535 


401 


C T 0 

512 


33/ 


25 


5 


415 


413 


OA/ 

364 


c 1 c 

515 


62 /• 


26 


5 


429 


394 


447 ' 


37o 


"7 £ 0 

7 63 


27 ' 


. 4 


554 


604 


620 


T C C 

755 


0 T O' 

3/9 


"28 


d 3 


753 


"729 


O-O r\ 

o'30 


"7 0 0 

72o 


461 


29 


.2 


356 


572 


529 


0 0 y 

294 


OCT 

3o/ 


30 


5 


491 


592 


470' 


541 


COO 

50o 


31 


1 


679 


681 


713 


^00 
608 


475 


32 . 


4 


'620 


' 661 


641 


692 


0 T1 

271 


33 


2 


556 


724 


628 ^ 


46o 


611 


34 


1 


728 


• 661 


552 


T 0 1 

721 


00/ 
334 


35 


4 


638 


530 


571 


721 


110 
llo 


36 


2 


571. 


713 


'655 


/ 0 "7 

497 


00/ 
234 


37 


2 


418 


473 


386 


oil 

311 


Tin 

^ 110 


.38 


3 


720 


811 


890 


Ibb 


* 000 
.393 


39 


2 


677 


711 


633 


coo 

59o 


/TO 

412 


40 


.:'4 


611 


- 394 


Iff' 
Abb 


^00 


^ 0 
69 


•41 


4 


. 800 


111 


786 


761 


000 

332 ^ 


42 
43 • 


2 


552 


764 


^] i 


A c 1 

651 


0^0 
369 


r 1 ' 


401 


513 


5^5' 


390 


517 


.44 


1 


377 


.421 


518 


339 


^•118 


45 


. 1 


591 


417 


"342 


° 416 - 


4 


46 


3 


535 


506 


519 


503 


437 


47 


2 


679 


686 


731 


756 


353 


48 


4 


638 


556 


638 


742 


23 


49 


5 


467 


391 


465 


■ 416 


475 


50 


1 


--426 


-446 


-458 


-266 


-185 



*This table includes information only on the final 100 items selected. 
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TABLE 2 (cont.) ' , 

Each Item Correlated With Total Scale Scores for Five Scales. 
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Scales 



Item 


Scale 


1 


2 


3 




5 




Number 


Number 












51 


1 


484 


' 549 


534 . 


437 


O "7 C 




52 


2 


623 


784 


716 


664 


c oc\ 

Do9 




53 


• 1 


506- 


317 


199 


366 


Q tr n 




• 54 


4. 


670 


' 825 


803 


Ikl 


439 




55 


*. 5 


157 


243 


375 


^ 218 


^ o o 




56 




5 .4 


393 


296 


365 


1 "7 A 




57 


1 


'^o8 


482 , 


556 


371 


o o o 

jOo 




58 


1 


423 


443 


366 


454 


35U 




, 59 


.1 


754 


570 


626 


694 


IOC 

135 




60 


4 


650 


595 


^24 


670 


1 o o 

loo 




61 


i 


703 


513 


480 


620 


36 




62 


3 


708 ' 


750 . 


777 


^ 677 


OTA 

il4 




■> 63 


.2 


V 600 


717 


635 


571 


494 




64- 


5 


168 ■ 


233 


379 • 


211 


587 




65 


2 


751 


784 


764 


_799 


57 




66 


3,. . 


jm 


486 


517 


. 476 


334 




1. 67 


3 


■ 384 


520 


648 


432 


o o r4 

?oO 




68 


■ 4- 


560 


483 ' 


387 


687 


138 




69 


5 


107 


197 


339 


86 


coo 

523 




.70 


2 , 


332 


462 


329 ■ 


306 


COT 

527 




71 


3 . 


478 


565 


533 


411 


o o 

36 o 




72. 


U 


581 


420 


599 


554 


o <o 

209 




73 


5 


" 423 


533 


445 


332 


733 




74 


4 


.550 


65.1 


567 


709 


506 




75 


4 


• 606 


510" 


446 


722^ 


7o 




76 


• 4" 


454 


395 


471 


693 


382 




11 


3 


. 790 


.646 


673 


742 


o <o o 

308 




,78 


2 


786 ■ 


794 


828 


784 


486 




79 


„ 1- 


790 


685 


586 


747 


390 




80 


■ 5 ^ 


482 ^ 


594 . 


612 


478 


£i C 

ODD 




81 


5 


300 


340 


274 


166 


644 




82 


2 


520 


527" 


444 


443 


o / o 

243 




83 


5 


. 550 


437 


423 


516 


^ o o 




84 


3 - 


^ 328 \ 


230 


394 


313 


-7 




85 


4 


566 


632 


579. 


59.2 


"7 O "7 
III 




86 


4 


560 


486 


488 


606- 


O O £ 

386 




87 . : 


1 


815 


.795 


781 


' 828 


591 




88 


4 


837 


878 


,904 


871 • 


c o c 

535 




,89 


4 . 


r.. 543- 


525 \ 


448 


625 


447 




90 • 


1 


613 


.666- ^ 


\ 764 


721 


407 




■ 91 


5 


409 


433 


♦ \^74 


366 


£i £i "i 

667 




92 


2 


334 


396 


337 


316 


694 




93 


3 


697 


695 


7 3 Ox 


559 


257 ' 




94 


1 


'360 


456 


420 ^ 


\ 286 


' 244. 




. 95 


1' 


/ 775 
670 


765 


831 


\ 670 


392 




96 


3 


747 


751 


\743 


503 




97 


5 


344 


451 


328 


25.3 


668 




98- 


■' 5 


378 


345 


323 


38\ 


. 350 




• •' 99 : 


3 


578 


632 


728 


. .623 \ 


402 




100 


2 


536 


688 


^604 


543 


\ 342 
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TABLE 3* 

Each' Item Correlated Witli Total Score of Instrument and Total Scale Score. 



Item 
Number 


K (.Scales 


(Total) 


Item 

•Number 
J 


IS. \ DCai-e/ 


Iv l^U Lcl J. / 


1 


r ■ 

7759 


.6724 




- 51 


.4840 


.5189 


2 


.4291 


. 5127 


52- 


- .7844 


.7775 


• 3 


. 6944 


.6099 


53 


.5058 


.3886 


■ -4 


.7097 


,6479 


54 


.7421 


.7932 


5 


. 8246 


.8203 


55 . 


.6279 


. 3472 ' 


6 


.7418 


.8044 ■ 


56 • 


.3932 


.3442 


7 


.6524 


•54,62 


57 ■ • 


.4824 


^ .4705 


8 


.5919 


.5936 


58 


. .422S( 


.4588 


9 


.8084 


.7231 


' 59 


.7535 


.6471 


' 10 • 


.4224 


.4712 


60 


.6B95 


..6319 


11 


.7688 ; 


.7402 


61 


.5132- 


.5513 


l'2 


.5080 ; 


.5673 


62 


.7774 


.7404 


13 


.6394 


-.6137 - 


r 63 


-.7165 


.6789 


U 


.8416 


. 76-57 • 


64 


.5868 


.3397 


. 15 


.3933 


. 3808 
.2232' ' 




65 , 


'. 7843 


.8319 


16 


.6379 




66 


.5166^ 


.5i59 


■ • . 17 


,6698" ' 


.6873 


67 


.6479 


.5176 


18 . 


• .6342 


' .6273 


68 


.6866 


.5249 


19' • 


.6910 


.■3922 


., 69 


^.5233 


.2651 


20 


.6452 


,6692 


70 


.4j619 


.4271 


21 


.7282 


.6436 • 


71 


.5334 


.5305 


22 


.6703 


.6330 


72 


..5537 


.5469 


23 


.6894 


. . .6432 


73 


.7325 


.5345 


24 


.5349 


.5;399 


74 


.7092 


.■6747 


25 


.6268 


.5149 


75 


.7221 


. .5524 . 


26 


.7634 


.5329 


76 


.6933 


. 5480 


27 , 


.7549 


.6672 . 


77 .. 


.6733 


.7265 


28 


.8297 


.7980 


78 


.7938 


. .8378 • 


N 29 


.5725 


.4771 


79 


.7905 


.7293 


30 


.5083' 


.5820 


80 


• ,6658 


.6271 


. - ' 3-1 


.6792 


.7142 




81 


.6442 


.3651 


32 


.6916 


.6643 


82 


.5267 


■ ,4957 


33 


. .7245i 


.6636 


' 0 J 


.6332 


.5662 


■ '34 


.7284 


.6855 


84 


,3937 


.2984' 


. 35 


.7214 


- , .6029 ; 


85 


.5923 


■ .6866 


. 36 


.7126 


.6111 


'• 86 


.6065 . 


.5743 


37 


. .4729 


.3895 


87 ■, 


.8146 


,8630 


38 


' .8902 " 


. \ .8208 


88 ' 


.8714- 


.9170 


, 39 


.7108 


. .6880 


89 


.6253 


-5844 


'40 


' .6385 


. .5122 - 






.6235 • 


.7278- 




—90- 


41 


. .7607 


^ .7936 


91 


.6666 


.5138' • 


42 


.7638 


.6715 


92 


".6942 


. 4476 


43 


* .4007 


.5261 




93 , 


.7301 


.6742 


44 


.3773 


.4103 


Q /• 






45 


.5906- , 


.4137 


95 . 


•.7751 


.7842 


46 • 


.5185 


.5627 


96 


.7510 


.7754 


47 


,,6864 


.7357 


97 ■> 


.6681 


.4386 


48 


. 7417 


.6135 


• 98 


.3504 


. 398,7 • 


49 


"747^6 


- .4945 1 




99- 


.7284 


.6757 


•• 50 


-.426>.^ 


• ' -.4051 ■ 




100 , 


^- .68a2 


.6167 , 



"0 *This 
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table includes informatipn only c 



n tive final 100 items selected. 
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THE '^TROUBLE SHOOTING" CHECKLIST (TSC) 
FOR HIGHER-EDUCATIONAL SETTINGS 
(Ijnstruaient) 



/ 
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The "Trouble Shooting" Checklist (TSC) 
For Higher-Educational Settings 

» • . <'-• 

Introduction arid Instructions 



. The TSC consists of lOD Likert-type items describing departmental charac- 
teristics. , These items are randomly arranged and can be broken into five 
scales focusing. on particular organizational variables which affect the adop- 

tion-dlf fusion process . The history of. t^e development of the instrument is 

^ ■" ... 

included in the manual, as- well as inf ormation'^oit uses of ^ the. TSC, scale;^ titles, 

numbers of items in each scale, and- scoring instructions. 

/The respondent is asked to rate whether "pr ^not'. an item is 'SescriptU-ve 

( tj^p/icalAa typical )■■ of a particular departmene^ Although there are descriptive 

staiLementB jabout individuals^ outside of the department <such^ as the dean), the ^ 

staltements, nevertheless, focus on such persons in relation, to the department. 

Th€f department being rated^shoul-d always be t^he 'first point of reference. 

■* ' , " ' ' ■ » - ' 

The term change agent is used frequently throughout the instrument. This 
. ■> • 
tejrm is used broadly and includes both' external change agents, (individuals 

bijought in from outside the department specifically to facilitate chahge) and 

/ ' ' ' ^ ; 

internal chang'4 agents (facilitators of change who are permanei^t members of the 

department). The role of the change a^ent may range from assisting a department 

in problem-solving by giving of their own prof ei?siotial expertise, to pteviding 

" . ■ ■' ■ * . 

contacts with all available resources, to actually taking part in decisions to 

adopt innovations and aiding in the implementation of adopted . innovations . It 

. • * ■ \ ' ' 

is al9o' assumed' that the department you arevrating is at least considering the 
adoption of one or more innovations (as reflected in items referring lo "inno-^ 
vation") ' . . . . 
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In order to complete th^instrument , merely rate on a 1 - 5 scale (as indi 

cated below) how; closely ^ach i^em describes the department you are i>ating and 

record your ratings on the line directly to tHe lef t -of ""the Atein. 

5 = very typical ° 
• 4 = somewhat typical ' ' • 

. 3 = neither typical nor atypical 

2 = somewhat atypical . ** ^ 

'1 = very, atypi'cal > ' . ^ 

In adjdition, you may^ use one of the following two ratings: . i 

1. If the item is applicable to "'the department yoii are rating but you 
do not ow the information » use the symbol. " . 

2. 'If . the item is not applicable . to th^ particular department you are / 
rating, . use the symbol "NA,'' 
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The "Trouble Shooting'^ Checklist (TSC) , 
For Higher^^*itational Settings. 



Please rate on a 1 ^ 5^ .scale (as indicated below) how closely each item describes 
the . d'Bpartment you are rating, or use one of the alternative syTiib^lLs: 

5 ^ very typical . . . 

•' 4 = somewhat typical . \ 

3 =' neither, typical nor atypical -r^ 
\ ' ^ 2 = somewhat atypicial " - . 

. ^ ;^ * ^ 1 = very atypical • ■ ^ ^ 

NA = not applicable . • i ' ' ^ ^ . 

? .= no information at this time' , **' 

^ ^ 1. The students* complain that tfie courses are unj^i^^ed to their personal 

and/or professional goals. ^^^''^ - ^ 

2. Students ideas have been acted on in the past, v » 

3.' The faculty are no.t able 'to talk abou,t the sJ^stance of particular 

innovations, though they may be^able to name ^me innovajt 

^ 4.^ The faculty do ni|t systemat,ically 3eek out approaches outside of their 

orientation. . , 

5. Th^ departments chairperson appears uncomfortable when changes |in the 
department are discussed," ^ ^ 

, 6. The few faculty members who appear to be innovative do not communicate 

weH with the .rest of the faculty, • * . ' 

It will be necessary to convince'a doubting faculty of th^ admit^istra-' 
tion's interest in innovation. . ^ 



partdJcuJ 
ijtional 



7. 



8. The department chiaii-per son Is concerned with current developments- rele- 
vant to an innovation under consideration for adoption/ 

• ■ ^ , • / 

9. All of the faculty members seem equally involved in . increasing the ; 
level of use of previously adopted innovations. 



.10. The students are in frequetlt contact with one another (e.g., in semi- 
nars, in th^ field, in the learning resource centeV, ^tc), 
' ' -J 

11. There are pnly a few facility members who are trying to arouse interest 

in actual trial testing of an innovation. 

12. Members of the department take responsibility 56r their decisions and 

actions . . . / * . 

J 13. iChis department is only interested in the expenditures associated with 
adoption of particular innovations. / 
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Please rate on a 1 scale (as indicated below) how closely ea^h item describes 
th^ (department you are rating, or use one of the alternative , symbols : 



5 = very typical 
4 == somewhat typical \ 
3 = nei;ther typical nor atypical 
% = somewhk^'* atypical 

1 = very atypical * 
. ' NA = not .applicable ^, - ' ' ^ ' 

?. = no -i^nforination at this time 

— • * ** 

14. .The students in this department read suggested materials in addition to 
* the required reading assignments. . . 

15. The faculty ask, reflective and/or analytical questions about the adop- 
tion/ implementatiofi process. ' 

16. This, department has developed many 9 f its own products,. 



_ 17'. Communications between a change agent and the department, focus on how 
n^ practices aid in effective teaching. 

^ 18. The students are not well acquainted with each other. 

19. The department " is involved with the successful adoption of other inno- 
vations. 

20. Communications concerning innovation have^all been enthusiastic and 
positive . * 

21. The students realistically assess their needs. 

22. The faculty members who have some interest in innovation have no pres- 
tige or tenure in the department. - , ' 

23. This department gives the faculty financial support for educational 
expenses related to the adoption of innovations, 

_ 24. Although material on innovations is mentioned by the faculty from time 
to time, 'it has probably not been read by anyone, 

25. The students claim that they are intellectually challenged by changes 
which have been made in the department's teaching approach. 

26. The students take initiative in seeking out challenging course work. • 

27. Some of the faculty arfe asking for detailed information about the ^ 
V mechanics of u^irrg^an innovation. 

28. Members of the faculty have refused to discuss even tlie possibility of 
taking on new roles within the department. 
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Please rate on a 1 ~ 5 scale (as indicated below) how closely each item describes 
the department you are rating, or use one of the alternative symbols: 

. '5 = very typical " - u - 

4 = somewhat typical 
3 = neither typical nor atypical 
2 '= somewhat atypical ^ . . 

1 = very atypical ,v^. • • ^ 

NA = not applicable 

. ? == no information at this time ' 



29. There is a small faculty faction trying to replace the department chair- 
person. . ' " \ '. 

30. The supporters of innovation are not normally the initiators of, coramuni- 
cations'" within this, department. 

31. The faculty receives most communications from administrators by way of^ 
memo.- • ^ . 



32. -The faculty members arrive at school late^ 



teave early. 



33. The change agentjiOT 
autl ' ^ 



at this. .department is not in a position of 



34. This department"^" does not hire. faculty who have had experience in devel- 
oping innovations. • 

35. The faculty listen to the suggestions "and. ideas of students. 

'36. The supporters of innovatiori'.have serious communication problems with 
the faculty at large. * ' 

37. The de^an is unaware of recent changes which have been made within fhis 
department. 'i , - - \ 

38. Gommunic.ations between department members and change agents result m 
constructive actions within the department.!; 

39*. This department feels comfortable coramunioating often with: change agents, 

40. The faculty is warm and personable. 

41. 



42. 




Faculty members of this department have already established" general ' 
strategies for implementing innoyations^j^ ' j 

Some meTrtbers of^jAe-'depaTtment have made efforts to communicate with 
£e&agn±Ze3^xpert3 in their subject areas. — - 



This department has thus far made only minimal efforts in seeking out 
assistance in making changes. *■ . 
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■h ' - . ■ ^ 

Please rate on a 1 - 5 scale (as indicated below) how closely each item describes 
the department you are rating, or use one of the alternative symbols: 

• ' 5 = very typical 

4 = somewhat typical 
3 « neither typical nor atypical 
^ 2 = sdmewhat atypical 
1 = very atypical 
NA = not applicable 
? = nd information at this time^ 

..„ ; . ^ -i\ 

\ 

. 44. The dean^ is not^^^-S-er^ve in establishing goals, 

45^^eTnBers of the faculty have stated that they are more concerned with 

" ^ effective teaching than advancement in the department hierarchy. 

46. This department is unaware of basic information about innovations. 

47. The department chairpefson uses many cliches (e.g., why change for the 

\ sake of change? before we buy any program, we must establish a sound 

philosophical basei etc.). > 

48. The dean acts as a hindrance to 'the adoption of innovations. 

49. The students are constantly exposed to new ideas in their coulrses. 

50. This department uses, resource materials effectively to develop its own 

materials. " : 

• 5i. The faculty frequently raise the issue of "standards^. " 



52. The faculty makes no attempt to encourage more student dnvolVement. 

■ . ■ ■ \ 

53. The faculty members frequently discuss how their Interpersonal rela- 
tionships with their colleagues affect the* functioning of th^ir^ programs. 



54. Members of the faculty freely, discus^s with each other the problems that 
^ they have in their- day to 'day work. \ 

35. .The students talk only aboqt "getting through" their tS^urses*' 

" • \ ' * 

56. The students have many shared experiences outside of the^ department . 

' \- • • 

57. Very few faculty members communicate with other faculty outside their 
- department about their work^ 

58. There is much discussion about . the impact of innovations on the educa- 
. tion of students. ; . 

. ' I ' ' ■ \ ' 

59. The faculty members express a desire to increase their understanding of 
both themselves and others. 
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Please rate on a 1 5 scale (as indie a^tedJiaiowT how closely each item describes 
the department you are ratin^j_jMe--trs'e''''o^ of the alternative symbols: 

5 = very typical 

4 = somewhat typical- 

3 - neither typical nor atypical 

2 = somewhat atypical ' 

1 = very atypical 

NA = not applicable 

? = no information at this time 



The department has no recognised leadership. 

Some of the faculty may already be committed to traditional teaching 
methods . 

The administration is concerned with adopting a program which will best 
meet the. students ' educational needs.- ♦ ^ - 

There is' a willingness to initiate needed change. 

Most innovation-related communications between faculty members consist 
pf remarks about the financial- situation and philosophical bases. 

This department often does not respond to cbmmunications from change 
agents . ' . - • 

Individual members of the department are not in a position to reinforce 
each o"ther.. ' 

The students relate to faculty members bnl5^ in the classroom. 

Innovation supporters and non-supporters have. emerged in the form of 
in- and out-groups. 

Change agents and the department have established a cQmfortable rapport 
over a period of time.. 

The department chairperson strongly supports change efforts (e.g., 
through public statements, promotion rewards, provision of resources, . 
etc.). 

The faculty members meet often to exchange ideas with one another. 
The faculty is actively developing innovations. 

The department chairperson does not "respond to requests from the 
faculty. . ' 

The faculty members ask questions about how innovations . can bring about 
specific changes in their department. ^ 



^ 62. 

^ 63. 
^ 64. 

^ 65. 

'/ 

_ 66. 

^ 67. 
^68. 

_, 69. 

70. 

— c 

71. 
_ 72. 
73. 



74, 
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Please rate on a 1 - 5 scale (as indicated below) how closely each item describes 
the department you are rating, or use one of the alternative symbols: 

5 = very typical 
J " 4 = somewhat typical 

3 = neither typical nor atypical 
2 = somewhat atypical , 
1 = very atypical 
NA = not applicable . 
' . ? = no information at this time * 



75. There has been little reinforcement from the department for fully imple- 
menting adopted innovations. - , . 



76. ''Many of the faculty, while not actively opposed to innovation, will not 
commit themselves. ^ „ 

77. The students praise their program for the interrelatedness of its 
courses . 

78. Older faculty members discourage younge'r . faculty members from remaining 
in the department. 

79. The administration listens to dissenting views. 

80. The only action in^ this department consists of classes and scheduled 
•office hours. - 

sr. The classes in this department are relatively large. 

82. Some individual faculty members are using an innovation comfortably. 

83. Interested faculty members are in frequent communication with change 
agents. . \. 

84. The structure of the organization includes reasot.ably well-functioning 
communicatioa channels . ^ ' 

85. Any initiative taken by change agents in this department is considered 
harassment by the rest o% the department. 

86. The students complain that their courses lack interrelatedness. 

87. Faculty members have made 'only one or two inquires about innovations. 

88. The students are encouraged to develop their own -styles. 

89. The dean has worked with the faculty in the past in their efforts to 
implement innovations. 
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Please ratd on a 1 - 5 scale (as indicated below) how closely each item describes 
the department you are rating, or use one of the alternative symbols: 



5 




very typical • 


4 




somewhat typical 


3 




neither typical nor atypical 


^ 2 




somewhat atypical 


1 




very atypical 


NA 




not applicable 


? 




no information at this time 



The students are excited about innovative approaches which compliment 
their individual* learning styles. 

The students feel frustrated and/or disillusioned because of a lack of 
"standards" in their field. • 

The leadership in key positions rewards, conformity. 

The faculty is threatened by new approaches. • ./ 

This faculty is actively seeking information on innovations. 

The department chairperson is primarily concerned with trie, quality of 
instruction. 

-It is difficult for a change ageht to contact key persons within this 
department. 

.J 

The faculty only weakly endorses any basic change . within the department 



The faculty does not need. .,t 6 be proded into using innovative approaches 

In the past, faculty interest in an innovation has resulted in early 
plans for pilot testing of that innovation. 

100*- Communications between a change agent and this department do not focus 
on real issues or problems. 



90. 

91. 

'92. 
93. 
94- 

95. . 

96-. 

97;. 
98. 
99. 
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